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Abstract. One of the objectives of this paper is to investigate the pedagogical impact of 
both the mobile testing system PeLe (Norway, HiST) and the enquiry-based learning 
approach on language skills development in the context of mobile-assisted learning. 
The research aims to work out a methodological framework of PeLe implementation 
into the language classroom through formative assessment, immediate feedback and 
interactive post-test activities. The framework was developed and pilot tested in a 
joint research project (MobiLL) by English as a foreign language (EFL) teachers 
at Lomonosov Moscow State University (LMSU, Russia) and University College 
HiST (Norway) during two semesters of the 2013-2014 academic year. Students 
enrolled in a preparatory English course at LMSU were randomly assigned to 2 
experimental groups and 2 control groups. Students in the experimental groups 
took a series of PeLe supported grammar and vocabulary tests as volunteers using 
handheld devices. The control groups were tested by the traditional testing method 
- pen and paper. The analysis based on quantitative research data demonstrated that 
PeLe supported language classes resulted in language skill gains. Qualitative data 
analysis highlighted the positive effect of mobile formative assessment and of post- 
test activities on learner motivation and collaboration skills. This study suggests that 
the use of technology was effective in engaging students in enquiry-based tasks, to 
produce more output in the target language. 
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1. Introduction 

Mobile devices have become “pivotal in students’ everyday life and mobile 
technologies are expected to play a bridging role between informal and formal 
practices of learning” (Mobile Education, 201 1, p. 12). In spite of the plethora of 
research in the area of mobile learning, it challenges instructors to examine how 
mobility relates to their teaching aims, methods and subject matter, because there 
is not yet consistent Mobile Assisted Language Learning (MALL) methodologies. 
On the other hand, there is also a need in the new educational framework for mobile 
testing tools implementation aimed at developing learner skills rather than just 
assessing learner knowledge and proficiency (Arnesen, Korpas, Hennissen, & Stav, 
2013). Hypothesis to guide the framework of our research includes collaborative 
enquiry-based learning and peer learning approaches and educational opportunities 
provided by handheld devices such as immediate feedback, interactivity and 
flexibility. This paper, supported by both current m-learning theory and enquiry- 
based learning theory, focuses on working out a methodological framework of 
mobile testing system implementation into the language classroom. 

2. Method 

2.1. Research objectives 

Mobile Language Learning (MOB1LL, http://histproject.no/node/859) is an 

international project involving two institutions: Sor-Trondelag University College 
(HiST, Trondheim, Norway), Department of Technology, and Lomonosov Moscow 
State University (LMSU, Moscow Russia), Department of Foreign Languages and 
Area Studies. The project was conducted during two periods from September 2013 
to May 2014. The data discussed in this paper are related only to the first period of 
the project. The key objective of this international research project was to work out 
sound pedagogical strategies on how to implement the mobile testing system PeLe 
into the traditional language classroom (pedagogical perspective), thus introducing 
some improvements to the piloting tool (technological perspective). 

The project target group consisted of Norwegian and Russian learners of English, 
all at the same language level (Bl). For the first period of experimentation, several 
classes of learners at LMSU participated, namely: students (n=35) enrolled in a 
preparatory English course at LMSU were randomly assigned to 3 experimental 
and 1 control groups. The control group was tested by the traditional testing 
method, i.e pen and paper. In this paper, data collected only from the LMSU groups 
will be discussed and analyzed. 
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The methodological framework of the MALL model based on PeLe implementation 
includes both enquiry-based methods such as collaborative and peer learning post- 
test activities, brainstorming, problem solving activities, group discussions and 
mobile learning opportunities such as immediate feedback, formative assessment, 
interactivity and flexibility. The project implementation and research design is 
illustrated in Figure 1. 


Figure 1. Methodology of PeLe implementation 

Methodology of PeLe intervention: Cycle 1 



2.2. Data collection 

Data collection was done in three cycles: 

1 . Intervention of PeLe tests (8 tests) as formative assessment tools from September 
to December 2013. The grids (24) on post-test activities used after each test were 
completed by the teachers of the experimental groups. Quantitative data were 
analyzed by mean (M) and standard deviation. 

2. Quantitative data of the final tests were gathered in control and experimental 
groups at the end of the semester, the data were analyzed by mean (M) and standard 
deviation. 

3. In January 2014, the post-intervention questionnaire asked students to reflect 
on their experience and attitude to the MALL model. Qualitative data were then 
gathered to help explain quantitative findings. 
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In the first cycle, students of the three experimental groups took a series of PeLe 
grammar and vocabulary tests (8). PeLe tests in the experimental groups were used 
for formative assessment and were provided in the form of in-class grammar tests. 
Students responded with their smartphones or tablets. They had access to PeLe 
tests by using Wi-Fi in class. 

As previously mentioned in Titova (2014), average scores were included to 
compare the overall performance of the control and experimental groups after the 
implementation of the intervention. The intervention data were supplemented by 
student feedback gained from a post-study Google questionnaire. The post-study 
questionnaire contains 1 1 questions in the format of Likert four-level scale aiming 
to get student views on the strengths and weaknesses of PeLe integration. The 
questionnaire was completed by 28 students of the experimental groups. 

3. Discussion 

The data collected on the overall scores of summative tests in two groups during 
the fall semester of 2013-2014 - control and experimental ones - strengthen 
Titova’s (2014) results that introduction of PeLe supported approach helped 
improve academic performance of the experimental groups in overall results of 
the final test, whereas the control group demonstrated decrease in overall scores. 
The likely interpretation of this improvement could not be attributed specifically 
to PeLe-assisted testing. As found by Beal, Walles, Arroyo, and Woolf (2007), 
“[a]n alternate possibility is that students' performance might improve [also] as 
the result of a general effect of interacting with [mobile devices], for example, by 
increasing students’ attention to the material; prior work suggests that interaction 
alone may enhance learning” (p. 44; see also Titova & Talmo, 2014). In addition, 
students might improve because of taking tests regularly, immediate feedback 
and enquiry -based strategies used by the instructors. Student answers in the post- 
study Google questionnaire indicated that they had an overall positive outlook 
regarding the MALL model. Some participants commented on the challenging 
nature of weekly tests and post-test activities. However, they claim that this 
approach improved their overall satisfaction because it helped them get prepared 
for their entrance exams. 

4. Conclusions 

Since our conclusions are based on subjective and objective data , the study has 
some limitations that need to be highlighted. Firstly, the number of participants was 
small (N= 35), so their reflections may not be equally applicable to all mobile learner 
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perceptions. Another limitation is that PeLe enables the instructor to diagnose not 
only group performance but also results of each student because each individual's 
responses are also stored. As a result, an individual student's performance can be 
tracked across multiple sessions. Further research needs to probe the effect of 
PeLe implementation on individual performance of the learner. Thirdly, although 
mobile testing system PeLe holds promise, more research is needed to determine 
its effects upon developing not only grammar skills but also some other skills such 
as speaking, writing and listening. 

This research suggests that PeLe integration into language learning could be 
efficient, especially if combined with collaborative enquiry-based learning and 
peer learning approaches and the pedagogical potential provided by mobile testing 
systems such as immediate feedback, interactivity and flexibility. The experimental 
results suggest that the MALL approach, combining current m-learning theory and 
enquiry-based learning theory and formative assessment, is most advisable. We 
hope that our approach will provide some constructs for pedagogical thinking 
about enhancing MALL with new mobile-assisted assessment methodology. 
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